An image search engine provides a convenient tool for users to retrieve their desired images from the large amount of images on the Web. However, users often find it difficult to identify the interesting images in the returned results that typically are returned by the search engine due to the excessive amount of images contained in the returned results. One way to lessen the time involved for a user to find interesting images in the returned results is image search result summarization. In general, image search result summarization selects representative images from the returned results for presentation to the user and alleviates the need for users to browse each of the images in the returned results.
For example, consider the situation where a user issues a query “apple” and the search engine returns hundreds of images sorted by relevance. The images returned for query “apple” may range from the fruit apple to Apple Inc. products, and even to apple-shaped rock. It is quite inefficient for a user to browse each image in the returned results to find the desired images. Actually, when several topics for “apple” are presented, users are able to obtain their targets more conveniently.
There are a variety of image collection summarization (ICS) techniques that are effective in selecting representative images from an image collection. In particular, one image collection summarization technique for automatically creating image summaries from an image collection formulates the problem as an optimization problem. This technique takes the image coverage and diversity into consideration and then describes a greedy algorithm to solve the optimization problem. Another technique uses landmark summarization and employs K-Means to cluster the images into visually similar groups. Then the technique select images from the clusters according to some heuristic criteria including visual coherence and interest point connections. Yet another technique computes an optimal partition based on a mixture-of-kernels technique and uses a sampling algorithm to select representative images. Still another technique uses a greedy method to recommend canonical images. This technique first adopts visual words to represent the visual features in a scene and then iteratively select the images that cover the most informative visual words. Another technique clusters photographs by utilizing image content and associated tags to summarize general queries, such as “love”, “CLOSEUP” and so on.
Although the above image collection summarization techniques are effective in selecting representative images from a collection, they are not optimal to summarize image search results. This is due to several reasons. One reason is that an image search engine often returns some noisy images that ideally should not be contained in the summarization result. Thus, selecting images primarily by coverage and diversity (as most of the ICS methods do) is not a good strategy in noisy circumstances. A second reason is that these image collection summarization techniques tend to ignore image relevance and assuming the images in the collection are all relevant. However, the relevance obtained from the search engine is useful prior information for images to be selected as summaries. The third reason is that the image quality in the summarization result is desirable for a quality users' experience. Low-quality images are non-informative for users if they occur in the summaries, since users cannot get the “complex” idea from such small-sized thumbnails. Studies have shown that a user's experience significantly suffers from low-quality summaries and that most users cannot tolerate any thumbnail images with low resolution.